Benchmarking Combinations of Learning and Testing Algorithms for Automata Learning

Automata learning enables model-based analysis of black-box systems by automatically constructing models from system observations, which are often collected via testing. The required testing budget to learn adequate heavily depends on the applied and techniques. Test cases executed for (1) collect behavioural information (2) falsify learned hypothesis automata. Falsification test-cases commonly selected through conformance Active algorithms additionally implement test-case selection strategies gain information, whereas passive derive solely given data. In an active setting, such require external selection, like repeated extend available There exist various approaches testing, where interdependencies among them affect performance. We investigate performance combinations six algorithms, including a algorithm, seven performing experiments using 153 benchmark models. discuss insights regarding different configurations types systems. Our findings may provide guidance future users automata learning. For example, counterexample processing during strongly impacts efficiency, is further affected approach type. Testing with random Wp-method performs best overall, while mutation-based well smaller